Papers with generating captions
Do Large Multimodal Models Solve Caption Generation for Scientific Figures? Lessons Learned from SciCap Challenge 2023 (2026.tacl-1)
Copied to clipboard
Ting-Yao Hsu, Yi-Li Hsu, Shaurya Rohatgi, Chieh-Yang Huang, Ho Yin Sam Ng, Ryan Rossi, Sungchul Kim, Tong Yu, Lun-Wei Ku, Clyde Lee Giles, Ting-Hao Huang
| Challenge: | SciCap dataset launched in 2021 aims to generate high-quality captions for scientific figures. |
| Approach: | They propose to use the SciCap dataset to develop models for captioning diverse figure types across various academic fields. |
| Outcome: | The proposed models showed impressive performance on the SciCap dataset and in various vision-and-language tasks. |
The Devil is in the Distributions: Explicit Modeling of Scene Content is Key in Zero-Shot Video Captioning (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for zero-shot video captioning focus on one key aspect of the scene and ignore the rest of the visual input. |
| Approach: | They propose a novel textual prompting strategy for zero-shot video captioning that uses a category-aware retrieval mechanism to promote prompt diversity while ensuring visual relevance. |
| Outcome: | The proposed method outperforms existing methods on in-domain and cross-domain settings. |
Neural Caption Generation for News Images (L18-1)
Copied to clipboard
| Challenge: | Existing methods for automatic caption generation of images are lacking in the field of image-related applications. |
| Approach: | They propose a method for automatically generating captions for news images . they propose several deep neural network architectures built upon Recurrent Neural Networks . |
| Outcome: | The proposed method outperforms a traditional method on a BBC News dataset using automatic evaluation and human evaluation. |
SciCap: Generating Captions for Scientific Figures (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Low-quality captions are common in scientific articles and can decrease understanding . this paper aims to develop an end-to-end neural framework to generate informative, high-quality figure captions for scientific figures and charts. |
| Approach: | They propose an end-to-end neural framework to automatically generate captions for scientific figures from a large-scale dataset . they used figure-type classification, sub-figure identification, text normalization, and caption text selection to build models that caption graph plots, the dominant figure type. |
| Outcome: | The proposed model can generate high-quality captions for scientific figures and charts from a large figure-caption dataset from arXiv. |
VC-Inspector: Advancing Reference-free Evaluation of Video Captions with Factual Analysis (2026.acl-long)
Copied to clipboard
| Challenge: | Existing metrics for caption evaluation lack factual accuracy and limited context handling . VC-Inspector provides reproducible, fact-aware alternative that aligns closely with human judgments. |
| Approach: | They propose a lightweight, open-source large multimodal model for reference-free evaluation of video captions with a focus on factual accuracy. |
| Outcome: | Experiments show that VC-Inspector can generalize across diverse domains and improve on existing metrics. |